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ABSTRACT 

Sign  language  is  the  medium  of  communication  for 
the  hearing  impaired  people.  It  uses  gestures  instead 
of  sound  to  convey  meaning.  It  combines  hand- 
shapes,  orientation  and  movement  of  the  hands,  arms 
or  body,  facial  expressions  and  lip-patterns  for 
conveying  messages. 

Different  types  of  project  are  done  against  deaf,  mute, 
hard  hearing  people.  A  system  with  computer  human 
interface  is  proposed  for  sign  language  recognition. 
But  there  is  country  wide  variation  available  in  that 
project.  The  main  idea  of  this  project  is  design  a 
system  which  is  useful  for  communication  of  that 
people  with  outside  world  in  any  public  places,  so  that 
no  need  to  interpreter  in  public  places.  In  that  project 
we  need  the  isolated  images  in  the  form  of  database 
with  Indian  sign  language  of  numeric  sign.  A  regular 
camera  is  useful  for  acquiring  this  numeric  sign. 
Principal  Component  Analysis  (PCA)  is  used  for 
preprocessing,  in  which  the  removal  of  redundant  and 
unwanted  data  is  done. 

Keywords:  PCA  ,  SVM  classifier 

1.  INTRODUCTION 

Sign  language  is  a  language  for  the  people  who  are 
challenging  in  hearing  and  speech.  Despite  common 
misconceptions,  sign  languages  are  complete  natural 
languages,  with  their  own  syntax  and  grammar.  Signs 
are  made  of  units  referred  to  as  cheremes,  which  are 
any  of  a  small  set  of  elements,  analogous  to  the 
phoneme  in  speech,  proposed  as  the  basic  structural 
units  by  which  the  signs  of  a  sign  language  are 
represented,  and  including  the  hand  shapes,  hand 
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movements,  and  locations  of  the  hands  in  relation  to 
the  body  employed  in  a  particular  sign  language.  A 
sign  is  made  of  cheremes  and  any  two  signs  can  be 
differentiated  using  at  least  one  chereme.  There  are 
mainly  two  different  approaches  in  sign  language 
recognition  -  Glove  based  approach  and  vision  based 
approach.  Sign  language  is  helpful  with 
communication  between  deaf,  mute,  hard  hearing 
people  and  normal  people.  In  normally,  sign  language 
is  understandable  for  the  signer  and  the  person  who 
know  the  sign  language  but  it  is  so  much  difficult  for 
who  does  not  know  the  sign  language  or  meaning  of 
any  gesture.  In  M.  Geetha  and  U.  C.  Manjusha  [4],  the 
strategy  of  the  Eigenvectors  method  consists  of 
extracting  the  characteristic  features. 

Where  the  images  of  Indian  sign  language  can  be  used 
as  input  and  the  system  will  display  the  English 
alphabet,  which  the  mute,  deaf  people  want  to  tell.  A 
regular  camera  is  used  for  acquiring  this  sign.  For 
developing  this  project  we  need  numeric  sign 
database  with  26  English  alphabets  sign  &  9 

Numeric  sign  with  proper  images.  In  that  project 
every  word  or  every  alphabet  is  assigned  with  a 
particular  image.  This  image  is  in  the  form  of  .jpg  so 
that  we  can  easily  use  the  database.  These  images  are 
in  static  or  dynamic  form.  Static  gestures  dynamic 
form.  Static  gestures  have  fixed  position  of  hand 
whereas  dynamic  gestures  have  movement  of  hands 
and  body  parts. 

The  proposed  scheme  uses  PCA  for  feature  extraction. 
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Principal  Component  Analysis  (PCA)  (Rafael  and 
Woods,  2002)  is  a  well-known  and  one  of  the  most 
successful  techniques  used  in  image  recognition  and 
compression  for  extracting  feature  and  representing 
data.  It  is  technique  widely  used  in  the  area  of  pattern 
recognition,  computer  vision  and  signal  processing. 
Principal  component  analysis  (PCA),  also  known  as 
Karhunen-Loeve  expansion,  is  a  classical  feature 
extraction  and  data  representation  technique  widely 
used  in  the  areas  of  pattern  recognition  and  computer 
vision  such  as  face  recognition.  The  purpose  of  PCA 
is  to  reduce  the  large  dimensionality  of  the  data  space 
(observed  variables)  to  the  smaller  intrinsic 
dimensionality  of  feature  space  (independent 
variables),  which  are  needed  to  describe  the  data 
economically.  This  is  the  case  when  there  is  a  strong 
correlation  between  observed  variables.  By  discarding 
minor  components,  the  PCA  effectively  reduces  the 
number  of  features  and  displays  the  data  set  in  a  low 
dimensional  subspace.  In  this  study  the  feature 
extraction  algorithm  based  on  PCA  is  chosen.  The 
coefficients  of  these  methods  are  used  as  feature 
vectors  which  efficiently  represent  extracted  image. 

Cloud  computing  is  an  emerging  technology  in  the 
present  scenario  which  enhances  enormous  data 
storage  and  its  processing.  Various  issues  are  present 
in  maintaining  the  cloud  environment.  Some  among 
them  include  data  privacy  and  data  confidentiality 
faced  by  the  data  owners  while  storing  their  private 
data  onto  the  cloud.  The  domain  of  Data  mining 
mainly  helps  in  efficient  retrieval  of  the  content  stored 
in  the  public  space  using  various  retrieval  methods. 
Generally,  in  cloud,  the  data  privacy  is  provided  by 
outsourcing  the  files  in  encrypted  form  before  being 
uploaded.  This  creates  more  demand  in  server  side 
data  utilization  i.e.,  searching  on  encrypted  files  from 
the  cloud.  As  of  now  the  retrieval  is  made  productive 
through  keyword  based  search.  Most  recent  works 
focus  on  the  single  keyword  provided  as  queries  in  the 
search  process.  In  the  keyword  ranked  search,  it  had 
been  difficult  to  store  when  the  number  of  keywords 
are  more.  In  the  work  carried  out  below,  an  efficient 
and  reliable  multi-keyword  search  method  called  K- 
Gram  search  is  developed.  Using  K-gram  it  is 
possible  to  retrieve  more  than  one  file  i.e.,  related  files 
to  the  keyword  searched,  productively  relative  to 
misspelled  keywords  too.  In  addition  the  user 
behavior  is  too  monitored  from  the  start  for  enhancing 
keyword  search  and  it  also  ensures  proper 
confidentiality  via  access  confirmation  through  secret 
keys.  The  system  thus  tends  to  provide  more  chances 


for  tackling  data  privacy  and  confidentiality  issues.  It 
can  also  achieve  sub  linear  search  time  thereby 
increasing  the  query  processing  and  accuracy  in 
keyword  search. 

LITERATURE  REVIEW 

There  are  different  theories  used  for  Indian  Sign 
Convention  presented  by  different  authors.  The 
surveyed  literature  on  Sign  Convention  is  as  follows: 

In  Tamil  sign  letter  Balakrishnan  G.  Subha  Rajam  [1], 
Proposed  a  method,  which  was  recognizing  a  32  set  of 
combinations  &  1 0  for  each  up  and  down  position  of 
fingers.  This  method  is  used  for  the  up/down  position 
of  fingers  which  is  converted  into  decimal  numbers, 
this  number  is  recognized  in  to  the  Tamil  alphabet  A 
set  of  database  in  the  form  of  images  of  sizes  640x480 
pixels  are  captured.  Palm  image  extraction  is  used  to 
convert  RGB  to  grayscale  images.  The  experiment 
result  is  96.87%. 

For  static  and  dynamic  alphabet  sign  Rekha  J., 
Bhattacharya  and  S.  Majumder  [2],  proposed  a  system 
this  system  was  used  23  static  ISL  alphabet  signs 
from  40  different  signers  are  collected  as  training 
samples  and  22  videos  are  used  as  testing  samples. 
The  images  are  extracted  by  the  method  of  Principle 
Curvature  Based  Region  Detector. 

Multi  class  non-linear  KNN  are  used  as  sign 
classifiers.  The  experiment  result  for  static  94.4%  and 
for  dynamics  it  was  86.4%.  In  Indonesia  northern  Bali 
jungle  village  of  “Bengkala”  for  generations,  where  a 
high  percentage  of  residents  are  Deaf.  Bengkala  has  a 
higher  than  normal  deaf-since-birth  population  for 
over  seven  generations.  Today,  42  of  Bengkala's 
almost-3,000  villagers  have  been  deaf  since  birth. 

By  comparison,  about  two  or  three  births  per  1,000  in 
the  United  States  produce  a  deaf  or  hard  of  hearing 
child.  The  high  percentage  of  deafness  is  caused  by  a 
geographically  centric  recessive  gene,  called  DFNB3, 
which  has  been  present  in  the  village  for  over  seven 
generations.  Rather  than  ostracizing  deaf  residents, 
villagers  in  Bengkala  have  adapted  to  a  deaf  lifestyle. 
Throughout  the  village,  people  speak  with  their  hands. 
Kata  kolok,  known  as  "the  talk  of  the  deaf,"  is  a 
unique,  rural  sign  language,  independent  of 
international  or  Indonesian  sign  language. 
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III.  METHODOLOGY 

The  proposed  system  is  shown  in  the  figure  1.  If  no 
standard  data  set  is  available  to  experiment  on 
automatic  recognition  of  ISL  gestures  then  Two  data 
sets  of  ISL  character  signs  are  created.  First  set 
contains  gestures  belongs  to  single  handed  Indian 
Sign  Language  characters  and  the  second  set  contains 
double  handed  gestures  of  Indian  Sign  Language.  The 
details  of  acquiring  of  data  set  are  given  in  data  set 
creation  section. 


Fig.2  Input  image 

2]  Preprocessing 

After  collecting  the  database  from  user  we  need  to 
preprocess  those  images.  Preprocessing  images 
commonly  used  for  removing  low-frequency 
background  noise,  normalizing  the  intensity  of  the 
individual  particles  images,  Firstly  we  convert  RGB 
images  into  grey  scale  images  by  using  MATLAB 
(rgb  to  gray  converter).  This  will  convert  RGB  images 
to  high  intensity  Grey  scale  images.  In  this  step  we 
can  perform  segmentation  and  noise  removal 
operation. 

The  main  aim  of  pre-processing  is  an  improvement  in 
input  data  (sign  language  images)  that  data  suppresses 
unwanted  distortions.  Image  preprocessing  technique 
uses  the  considerable  redundancy  in  images. 
Neighboring  pixel  corresponding  to  one  object  in  real 
image  have  adjusted  some  or  similar  brightness  value. 
Preprocessing  consists  of  thresholding,  erosion  and 
dilation  of  collected  data  images. 


Fig.l  Overview  of  proposed  system 

1]  Input  Sign  language  Image  - 

Two  types  of  dataset  are  creating  for  this  experiment. 
In  this  dataset  26  images  of  single  &  double  handed 
images  are  captured  using  digital  camera.  These 
images  are  in  the  form  of  200X300  RGB  pixel  size. 
These  images  are  collecting  from  single  mute  male 
person  for  each  character. 

We  use  this  image  in  the  JPG  format  because  it  is 
very  easy  to  extract  the  image  in  different  hardware 
and  software  environment.  The  second  dataset  created 
for  this  experiment  are  images  for  each  number.  The 
memory  required  for  this  dataset  is  nearly  40  kb.  This 
image  is  flat  rectangular  shape  in  structure. 


i.  Skin  Thresholding  : 

Skin  detection  is  used  to  search  for  the  human  hands 
and  discard  other  skin  colored  objects  for  every  frame 
captured  from  a  webcam  shown  in  fig  3.  Image 
thresholding  is  used  for  extracting  the  significant  or 
desired  part  of  an  image  and  removing  the  unwanted 
part  or  noise.  The  point  operator  of  major  interest  is 
called  thresholding  which  selects  pixels  that  have  a 
particular  value,  or  that  are  within  a  specified  range. 
With  thresholding,  the  image  can  be  segmented  based 
on  the  color.  This  holds  true  under  the  assumption  that 
a  reasonable  threshold  value  is  chosen.  A  reasonable 
threshold  value  is  the  taken  from  the  histogram  of  the 
original  image.  The  point  operator  helps  us  to  find 
objects  in  a  picture  if  the  brightness  level  or  range  is 
known.  Hence  the  object’s  brightness  must  be  known. 
After  detecting  skin  area  for  every  frame  captured,  we 
used  contours  comparison  of  that  area  with  the  loaded 
hand  postures  contours  to  get  rid  of  other  skin  like 
objects  exist  in  the  image.  If  the  contours  comparison 
of  skin  detected  area  complies  with  any  one  of  the 
stored  hand  gesture  contours,  a  small  image  will 
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enclose  the  hand  gesture  area  only  and  that  small 
image  will  be  used  for  extracting  the  PCA  features. 


Fig.3  Threshold  image 


ii.  Dilation  and  erosion  : 

Dilation  and  erosion  are  basic  morphological 
operations.  They  are  defined  in  terms  of  more 
elementary  set  operations,  but  are  employed  as  the 
basic  elements  of  many  algorithms.  Dilation  is  used  to 
increase  the  object  size  where  as  erosion  is  used  to 
diminish  the  size  of  the  object.  Both  dilation  and 
erosion  are  produced  by  the  moving  the  mask  around 
the  image.  The  mask  which  is  also  called  as 
structuring  element  or  sub  image  or  kernel  (Lim, 
1990)  and  (Soille,  2004)  and  it  has  both  a  shape  and 
an  origin.  The  following  Eq.  1  and  2  are  generally 
used  for  dilation  and  erosion: 


Dilation  =  X  _  S  (1) 
Erosion  =  X  S  (2) 


Fig.4  Eroded  image  Fig.5  dilated  image 


Where,  X  is  an  original  image  and  S  is  a  structuring 
element. 

Based  on  the  image  which  is  to  be  dilated  or  eroded 
the  size  of  the  structuring  element  is  choose  it  is  a  odd 
square  matrix  which  contains  binary  elements  that  is 
0’s  and  l’s  and  also  the  dimension  should  not  exceed 
the  size  of  the  image  which  is  to  be  processed.  The 
structuring  element  is  a  set  of  coordinates.  The  origin 
of  the  structuring  element  is  the  centre  element  for 
most  cases  but  it  may  not  be  centre  point  for  less 
cases.  The  structuring  element  (Lim,  1990;  Soille, 
2004)  is  used  to  remove  the  eyelashes  and  eyelids 
whose  dimension  based  on  region  of  interest. 
Following  are  the  examples  of  structuring  elements. 
In  our  work  we  choose  the  3x3  structuring  element 
having  its  all  element  as  1 : 

111  010  110 

111  111  101 


111  010  101 

The  origin  of  the  structuring  element  identifies  the 
pixel  of  interest  in  the  input  image  and  all  the  pixels 
in  its  neighborhood  are  assigned  to  the  corresponding 
pixel  in  the  output  image.  The  result  of  Erosion  is 
subtracted  from  the  result  of  dilation  will  provide  the 
edge.  The  following  equations  are  used  for  dilation 
and  erosion  respectively.  The  result  of  erosion  and 
dilation  is  given  in  Fig.  4  and  5: 

3]  Feature  Extraction  - 

The  input  data  which  is  to  be  processed  is  transformed 
into  a  reduced  representation  set  of  features. 

This  is  referred  as  feature  extraction.  Every  image 
consists  of  large  amount  of  data.  This  information  can 
be  automatically  extracted  from  the  images  is  called 
as  feature  extraction. 

Here  we  are  using  Principal  component  analysis 
method  for  extracting  the  features. 

PCA: 

Linear  Discriminant  Analysis  (LDA),  Independent 
Component  Analysis  and  PCA  are  some  of  the 
techniques  used  for  feature  extraction,  among  them 
PCA  is  powerful  method  in  image  formation,  Data 
patterns,  similarities  and  differences  between  them  are 
identified  efficiently. 

The  other  main  advantage  of  PCA  is  dimension  will 
be  reduced  by  avoiding  redundant  information, 
(Daugman,1993)  without  much  loss.  Better 
understanding  of  principal  component  analysis  is 
through  statistics  and  some  of  the  mathematical 
techniques  which  are  Eigen  values,  Eigen  vectors. 
PCA  is  a  useful  statistical  and  common  technique  that 
has  found  application  in  fields  such  as  image 
recognition  and  compression.  Principal  Component 
Analysis  (PCA)  is  a  mathematical  procedure  that  uses 
linear  transformations  to  map  data  from  high 
dimensional  space  to  low  dimensional  space.  The  low 
dimensional  space  can  be  determined  by  Eigen 
vectors  of  the  covariance  matrix. 

PCA  Algorithm 

Following  are  steps  involve; 

Step  1:  Column  or  row  vector  of  size  N2  represents 
the  set  of  M  images  (Bl,  B2,  B3...BM)  with  size 
N*N 
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Step  2:  The  training  set  image  average  (p)  is 
described  as 


\H 


Bn 


m  n_, 


(1) 


Step  3:  the  average  image  by  vector  (W)  is  different 
for  each  trainee  image 


Wi  =  Bi  -  p  (2) 

Step  4:  Total  Scatter  Matrix  or  Covariance  Matrix  is 
Calculated  from  6  as  shown  below: 


M 

C  -  V  wnwnt  —  A  AT  , 

(3) 


Where  A=  [Wl  W2W3 . . .  Wn] 

Step  5:  Measure  the  eigenvectors  UL  and  Eigen 
values  eL  of  the  covariance  matrix  C. 

Step  6:  For  image  classification,  this  feature  space 
can  be  utilized.  Measure  the  vectors  of  weights 

QT  =  [w  1 .  w2,  wM],(4) 

Whereby, 

Hk  =  UkT  (B  -  p),  k  =  1,  2,  ...,M'  (5) 

4]  Feature  Vector  - 

In  machine  learning  and  pattern  recognition  feature 
vector  is  a  n-dimensional  vector  consist  of  numerical 
vector  which  represent  some  object.  Many  algorithm 
requires  feature  numerical  representation  of  object. 
When  representing  the  image  feature  value  shows  the 
pixel  of  image  or  whole  object  in  images.  Feature 
vector  are  equivalent  to  vector  of  variable  used  in 
linear  process.  It  is  just  a  vector  consisting  of  multiple 
element  or  feature.  Examples  of  features  are  color 
component  length,  area,  circularity,  grey  level 
intensity,  magnitude,  direction  it’s  depend  on  which 
feature  is  useful  for  application. 


Fig.6  Extracted  image 
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Fig.  7:  PC  A  Performance  Evaluation 
5]  SVM  Classifier  - 

There  are  many  types  of  classifier  Technique 
available  we  use  SVM  classifier  to  identify  the  image. 
SVM  is  a  support  vector  machine  used  for  supervised 
learning  model  with  associated  algorithm  that 
analyzed  data  used  for  classification  and  analysis  by 
using  Extracted  features.  SVM  classifier  is  the  method 
of  performing  the  classification  task.  The  support 
vector  machine  searches  for  the  closest  path  which  we 
call  “support  Vector”.  Once  it  found  the  closest  point 
then  the  SVM  draw  a  line  for  connecting  them.  The 
support  vector  machine  then  declares  best  separating 
line  which  bisect  and  perpendicular  to  connecting 
line.  We  perform  classification  by  finding  the  hyper 
plane  for  differentiate  between  two  classes.  Support 
vector  machine  are  simply  the  coordinate  of 
individual  observation. 

IV.  RESULTS  AND  DISCUSSIONS 

The  data  set  divided  into  two  groups,  one  used  for 
training  and  other  for  testing.  The  training  set  consists 
of  70%  of  the  aggregate  data  and  remaining  30%  are 
used  as  testing  means  for  training  we  are  using  images 
captured  from  6  different  people  and  for  testing 
images  captured  from  2  different  people  numeric  as 
well  as  alphabets.  We  also  perform  experiments  on 
same  (30%  or  70%)  dataset  which  is  training  as  well 
as  testing  for  SVM  classifier.  The  overall  accuracy  of 
the  proposed  method  as  given  below  Total  No.  of 
Patterns  -  No.  of  false  result  Patterns 


Accuracy 

100% 


Total  No.  of  Patterns  -  No.  of  false  result  Patterns 
=  .  X 


Total  No.  of  Patterns 


From  total  67  testing  images  64  images  are  correctly 
recognized  so  the  accuracy  of  this  proposed  method 
95.52%  using  PC  A  features  and  SVM  classifier. 
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linage 

Accuracy 

in  % 

100% 

iJ 

100% 

100% 

■ 

66.67% 

Image 


Accuracy  iu 
% 


66.67% 


100% 


Idl 


100% 


66.67% 


Tablel.  Accuracy  of  Sign  language 
V.  CONCLUSION 

This  Recognizing  system  is  capable  of  Recognizing 
Alphabetical  and  numerical  sign  with  high  accuracy 
using  SVM  (Support  Vector  Machine)  Classifier.  The 
experimental  result  is  shown  that  the  system  is  used  as 
“working  System”  in  regular  ISL  Recognition. 

FUTURE  SCOPE 

The  system  can  be  useful  for  ISL  Static  alphabetical 
and  numerical  sign.  The  system  is  not  useful  for 
complete  system.  For  complete  system  we  have  to 
include  ISL  word  or  sentences  in  future.  Also  other 
feature  extraction  algorithms  like  Wavelet  transform, 
Invariant  moments,  Shape  lets  descriptors  and  other 
existing  methods  can  be  included  in  conducting 
experiments  for  improvement  in  the  results.  Other 
classifiers  like  Principal  Component  Analysis  (PCA) 
and  Linear  Discriminant  Analysis  (LDA)  or  a 
combination  of  these  classifiers  can  be  included  in 
conducting  experiments  to  improve  the  recognition 
rate. 
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